Fully Automatic Analysis of Engagement and Its Relationship to Personality in Human-Robot Interactions

Engagement is crucial to designing intelligent systems that can adapt to the characteristics of their users. This paper focuses on automatic analysis and classification of engagement based on humans’ and robot’s personality profiles in a triadic human-human-robot interaction setting. More explicitly, we present a study that involves two participants interacting with a humanoid robot, and investigate how participants’ personalities can be used together with the robot’s personality to predict the engagement state of each participant. The fully automatic system is firstly trained to predict the Big Five personality traits of each participant by extracting individual and interpersonal features from their nonverbal behavioural cues. Secondly, the output of the personality prediction system is used as an input to the engagement classification system. Thirdly, we focus on the concept of “group engagement”, which we define as the collective engagement of the participants with the robot, and analyse the impact of similar and dissimilar personalities on the engagement classification. Our experimental results show that (i) using the automatically predicted personality labels for engagement classification yields an F-measure on par with using the manually annotated personality labels, demonstrating the effectiveness of the automatic personality prediction module proposed; (ii) using the individual and interpersonal features without utilising personality information is not sufficient for engagement classification, instead incorporating the participants’ and robot’s personalities with individual/interpersonal features increases engagement classification performance; and (iii) the best classification performance is achieved when the participants and the robot are extroverted, while the worst results are obtained when all are introverted.This work was performed within the Labex SMART project (ANR-11-LABX-65) supported by French state funds managed by the ANR within the Investissements d’Avenir programme under reference ANR-11-IDEX-0004-02. The work of Oya Celiktutan and Hatice Gunes is also funded by the EPSRC under its IDEAS Factory Sandpits call on Digital Personhood (Grant Ref.: EP/L00416X/1).This is the author accepted manuscript. The final version is available from Institute of Electrical and Electronics Engineers via http://dx.doi.org/10.1109/ACCESS.2016.261452

Impact of Tone-mapping Algorithms on Subjective and Objective Face Recognition in HDR Images

Crowdsourcing is a popular tool for conducting subjective evaluations in uncontrolled environments and at low cost. In this paper, a crowdsourcing study is conducted to investigate the impact of High Dynamic Range (HDR) imaging on subjective face recognition accuracy. For that purpose, a dataset of HDR images of people depicted in high-contrast lighting conditions was created and their faces were manually cropped to construct a probe set of faces. Crowdsourcing-based face recognition was conducted for five differently tone-mapped versions of HDR faces and were compared to face recognition in a typical Low Dynamic Range alternative. A similar experiment was also conducted using three automatic face recognition algorithms. The comparative analysis results of face recognition by human subjects through crowdsourcing and machine vision face recognition show that HDR imaging affects the recognition results of human and computer vision approaches differently

Identifying Emotions Using Topographic Conditioning Maps

The amygdala is the neural structure that acts as an evaluator of potentially threatening stimuli. We present a biologically plausible model of the visual fear conditioning pathways leading to the amygdala, using a topographic conditioning map (TCM). To evaluate the model, we first use abstract stimuli to understand its ability to form topographic representations, and subsequently to condition on arbitrary stimuli. We then present results on facial emotion recognition using the sub-cortical pathway of the model. Compared to other emotion classification approaches, our model performs well, but does not have the need to pre-specify features. This generic ability to organise visual stimuli is enhanced through conditioning, which also improves classification performance. Our approach demonstrates that a biologically motivated model can be applied to real-world tasks, while allowing us to explore biological hypotheses

Sentic maxine: Multimodal affective fusion and emotional paths

10.1007/978-3-642-31362-2_61Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)7368 LNCSPART 2555-56

Fully Automatic Analysis of Engagement and Its Relationship to Personality in Human-Robot Interactions

Engagement is crucial to designing intelligent systems that can adapt to the characteristics of their users. This paper focuses on the automatic analysis and classification of engagement based on humans' and robot's personality profiles in a triadic human-human-robot interaction setting. More explicitly, we present a study that involves two participants interacting with a humanoid robot, and investigate how participants' personalities can be used together with the robot's personality to predict the engagement state of each participant. The fully automatic system is first trained to predict the Big Five personality traits of each participant by extracting individual and interpersonal features from their nonverbal behavioural cues. Second, the output of the personality prediction system is used as an input to the engagement classification system. Third, we focus on the concept of 'group engagement', which we define as the collective engagement of the participants with the robot, and analyze the impact of similar and dissimilar personalities on the engagement classification. Our experimental results show that: 1) using the automatically predicted personality labels for engagement classification yields an F-measure on par with using the manually annotated personality labels, demonstrating the effectiveness of the automatic personality prediction module proposed; 2) using the individual and interpersonal features without utilizing personality information is not sufficient for engagement classification, instead incorporating the participants and robots personalities with individual/interpersonal features increases engagement classification performance; and 3) the best classi fication performance is achieved when the participants and the robot are extroverted, while the worst results are obtained when all are introverted

OverNet : lightweight multi-scale super-resolution with overscaling network

Super-resolution (SR) has achieved great success due to the development of deep convolutional neural networks (CNNs). However, as the depth and width of the networks increase, CNN-based SR methods have been faced with the challenge of computational complexity in practice. Moreover, most SR methods train a dedicated model for each target resolution, losing generality and increasing memory requirements. To address these limitations we introduce OverNet, a deep but lightweight convolutional network to solve SISR at arbitrary scale factors with a single model. We make the following contributions: first, we introduce a lightweight feature extractor that enforces efficient reuse of information through a novel recursive structure of skip and dense connections. Second, to maximize the performance of the feature extractor, we propose a model agnostic reconstruction module that generates accurate high-resolution images from overscaled feature maps obtained from any SR architecture. Third, we introduce a multi-scale loss function to achieve generalization across scales. Experiments show that our proposal outperforms previous state-of-theart approaches in standard benchmarks, while maintaining relatively low computation and memory requirements

Preface

Peer reviewed: YesNRC publication: Ye